On Multiple User Channels with Causal State 
Information at the Transmitters 
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Abstract — We extend Shannon's result on the capacity of 
channels with state information to multiple user channels. More 
specifically, we characterize the capacity (region) of degraded 
broadcast channels and physically degraded relay channels where 
the channel state information is causally available at the trans- 
mitters. We also obtain inner and outer bounds on the capacity 
region for multiple access channels with causal state information 
at the transmitters. 



I. Introduction 

In his 1958 paper [1], Shannon considered a communication 
system where additional side information about the channel is 
causally available at the transmitter. (See Figure 1.) He showed 
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Fig. 1. Communication channel with state information causally known at 
the transmitter. 



that the capacity of this channel can be achieved by adding 
a physical device in front of the channel, which depends on 
the current state (and the message to be sent) only. Using the 
modern language, we can write the capacity of this channel 

as 



C = max /([/; Y) 

p{u)p{x\u.s) 



max /(C/;y), 

p{u), x=f{u,s) 



(1) 



where U is an auxilliary random variable with finite cardinal- 
ity. The achievability of the rate in Q is clear from Shannon's 
original argument of attaching a physical device in front of 
the channel. The optimality of this rate is deceptively easy to 
show. Indeed, by recognizing Ui := (W, S*'^^) to be 

independent of Si and by observing that Xi can be written as 
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a function of {Ui, Si), we have 

i(W; y") = - ff(y"|w) 

i 

<Y,H{Y,) - H{Y,\W,Y'-\S'-') 

i 

= Y,IiU.r,Y,)<nC. (2) 

i 

It is also worth pointing out the similarity of Q to the capacity 
C when the state information is noncausally available at the 
transmitter, as shown by Gel'fand and Pinsker [2] and Heegard 
and El Gamal [3]: 

C'^ max I{U;Y)-I{U;S). 

p\u\s)p\x\u,s) 

Since the capacity C for the causal case can be written as 
maxp(„')p(j.|„ s) I{U; Y)—I{U; S), the loss of causality merely 
lies in the independence between U and S. 

In this paper, we try to extend Shannon's result to multiple 
user channels. Research in this direction is not new. Most 
notably, Steinberg [4] obtained bounds on the capacity region 
of degraded broadcast channels when the state information is 
noncausally available at the transmitter, although the tightness 
of these bounds is still open. 

Causality of the state information makes the problem much 
easier Recently, Steinberg [5] reported the capacity region of 
the degraded broadcast channel when the state information 
is causally available at the transmitter In Section II, we 
give an independent treatment of his result by scaling the 
converse technique we just developed for the single-user case. 
In Section III, we run the same program for the physically 
degraded relay channel and obtain the capacity. Unfortunately, 
the optimality of the similar coding scheme for multiple access 
channels is yet to be established, and we present inner and 
outer bounds on the capacity region in Section IV. 

II. Degraded Broadcast Channels 

Definition 1: The discrete memoryless broadcast channel 
with state information consists of input alphabet X, state al- 
phabet S, output alphabet J^i x 3^2, and a probability transition 
function p{yi, y2\x, s), as in Figure |2l 



We assume the state information S" is available causally to 
the transmitter. We also assume that the channel is physically 
degraded, i.e., 

p{yi,y2\x,s) = p{yi\x,s)p{y2\yi). 



thus have 
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Fig. 2. Broadcast channel with state causally available at the transmitter. 



Theorem 1: The capacity region of the degraded broadcast 
channel with state information available causally at the trans- 
mitter is the set of all rate pairs R2) satisfying 

Ri <I{UuYi\U2) 

R2<I{U2\Y2) 

for some joint distribution p{ui,U2)p{s)^x = f{ui,U2,s) 
where the auxiliary random variables Ui and U2 have finite 
alphabets. 

Proof: Although the proof of the achievability can be 
found in [5], we repeat it here for the sake of completeness. 

Fix i?2 and p{u2)piui\u2)p{x\ui,U2, s). 

Codebook Generation. Generate 2"^^ independent code- 
words ^2(^2) according to n"=iP('"2i)- for ^^ch of these 
codewords, generate 2"^^ independent codewords u"{wi, W2) 
according to n"=i p{''-i-ii\u2i)- Codebook assignments are fixed 
and revealed to the transmitter and the receivers. 

Encoder To send wi G {1,2, . . . ,2^^^"^} to receiver 1 
and W2 e {1, 2, . . . , 2"'''^} to receiver 2, we select the 
corresponding codewords ^2(102) and Ui{wi,W2)- At time 
i, upon observing the channel state Si, the transmitter sends 

Xi = U2a, St). 

Decoding. Receiver 2 declares that a message W2 was sent 
if there is a unique W2 such that 143 (^2) and ^re jointly 
typical; otherwise an error is declared. Receiver 1 declares 
that a message pair {wi,W2) was sent if there is a unique 
{11)1,11)2) such that Ui{wi,ijU2), U2{u)2) and 1/2 are jointly 
typical; otherwise an error is declared. 

Define the following events: 

E2^^^mi^),Y^)eAi"'>} 

i?M = {(c/^W,yi")eA("'} 

i?M,.={(t/2"W,t/r(J,^),yl")GA(")} 

Without loss of generality, assume the sent messages were 
wi ~ I and W2 — I and denote the probabilities that receivers 
1 and 2 declare an error with and P2"', respectively. We 



Hence, P. 



(n) 



Pt^ = P {E'2A\Ju,^iE2,t 
<P{El,)+Y.P{E2,) 

< e + 2"''^22-n(-f('^2;y2)-2e) 

if i?2 < /(C/2; ^2) - 2e. Similarily, 



= p [El,,, y u,^ii?i,, y u,^ii?i4,, 

By the asymptotic equipartition property (AEP), P{Ef 1 1) — > 
and 'Et^iPiEi^t) < 2"-«22-"(^(c/2;n)-2e) ^ if i?2 < 
I{U2;Yi). But we already have R2 < I{U2;Y2) and the 
degradedness implies /(C/2;l2) < I{U2\Yi). Similarly, we 
can show that 

P{Eixj) < 2-"(^(^i'^il^^)-3.)_ 

Thus Y.j^iPiEi,i.j) < 2"-f^i2-"(-f(^i^^il^=)-3<^) ^ if 
Ri < I{Ui;Yi\U2) — 3e, which completes the proof of 
achievability. 

Let us now prove the converse. We wish to show that given 
any sequence of {{2"^\2"'^'-),n) codes {X,{Wi,W2, S'), 
WiiYO, W2iY^)) such that 

Pj"' := P{Wi ^ Wi or W2 ^ 1^2) ^ 

for uniform and independent message indices Wi and W2, the 
rate pair (i?i, R2) must satisfy the conditions in the theorem. 
By Fano's inequality, 

< nRiP^''^ + 1 = nei„ 
HiW2\Y2'') < ni?2Pi") + 1 = ne2n 

since pj"^ > niax{P^'-"\ pj"-*}. Define the following auxil- 
iary random variables, Uu := (VKi, F/"^, 5'"^) and U2i 
{W2,Yl-^). We now have 

nR2 < H{W2) 

= IiW2;Y^')+H{W2\Y^') 
<IiW2;Y^')+ne2n 



= Y,I{W2;Y2t\Yi-^)+ne2n 

i=l 
n 

<Y,I{W2,Yi-^;Y2i)+ne2n 

i=l 
n 

n 

= E I{U2i;Y2i) + ne2n- 



Similarily, 

nRi <H(Wi) 



= IiWi;Yn + HiWi\Yn 
<I{Wi;Y{')+nein 
<I{Wi;Y{'\W2) + nein 

71 

= I{Wi-Yu\W2,Yl-^) + ntin 

i=l 
n 

<Y,I{W,,Yl-\S^-'-Yu\W2.Yr^) 



nein 



<Y,HUl^■,Yu\U2^) + neln■ 

We define a random variable Q independent of everything 
else, uniformly distributed over {1,2,..., n) and define \J\ = 

(0,C/iq), C/2 = (g,C/2Q), X ^Xq,S ^ Sq, Yi = YiQ, 
and Y2 = I2Q. Then, we have 

1 " 

i?i <-y2I{Uu;Yu\U2^)+eln 

i=l 
1 " 

= - y I{Uu■Yu\U2^,Q = i) + ei,, 

i=l 

= I{UiQ;YiQ\U2Q,Q) + ein 
<I{UiQ,Q;YiQ\U2Q,Q) + ein 
= I{Ui;Yi\U2) + ein, 

and 

1 " 

R2 < - y I{U2^■,Y2^)+e2n 

i=l 

< I{U2Q;Y2q\Q) + e2n 

< I{U2Q,Q:Y2Q)+e2n 
^ I{U2;Y2) + e2n. 

But it is see to see that {1/1,1/2) is independent of S, that X 
is a deterministic function of {Ui,U2, S), and that the joint 
distribution of {X, S,Yi,Y2) is consistent with the channel 
p{yi,y2\x, s). Thus, we have established the converse of the 
theorem. ■ 

III. Physically Degraded Relay Channels 

Definition 2: The discrete memoryless relay channel with 
state information consists of input alphabet X, relay in- 
put alphabet Xi, state alphabet S, relay output alphabet 
3^1, output alphabet y, and a probability transition function 
p{y,yi\x,xi,s). 

In the following, we will assume that the state variable S 
is causally available to the transmitter and the relay. We also 
assume that the channel is physically degraded, i.e., 

p{y, yi\x, xi,s) ^ p{yi\x, Xi, s)p{y\yi, xi, s). 

The main result of this section is the following: 
Theorem 2: The capacity of the degraded relay channel 
with state information causally available at the transmitter and 
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Fig. 3. Relay channel with state infomiation available causally at the 
transmitter and relay 



relay is given by 

C= max min{/(C/,;7i;y),/(?7; Yi|;7i,S')}. 

p(u,ui) 

Proof: We first prove the achievability of the rate region. 
The approach follows Shannon's method of attaching physical 
devices [1] and the relay coding theorem by Cover and 
El Gamal [6], which transforms the original relay channel into 
one with auxilliary inputs U and Ui. 

More specifically, we use the block Markov encoding. 
A sequence of _B — 1 messages, Wi S W, each selected 
independly and uniformly over W, are to be sent over the 
channel in nB transmissions. Within each block of length n, 
the sender and the relay use a doubly-indexed set of codewords 

C = -.well,..., 2"^}, 

t e {0,l,...,2"-"«}} 

Fix a probability distribution p{u,ui),x = f{u,s),xi = 
fi{ui,s). 

Codebook Generation: Generate at random 2"^" indepen- 
dent n-sequences u"{t), t G {1, . . . , 2"^°}, each drawn 
according to YVi=iPi'^ii)- Po'" ^^ch sequence, generate 

211-R conditionally independent w"(it;|t) sequences drawn ac- 
cording to nr=i This defines the random codebook 
C. 

For each message w G {1, . . . , 2"^} assign an index t{w) at 
random from {!,..., 2"^}. The set of messages with the same 
index form a bin % C W. The codebook and bin assignments 
are revealed to all parties. 

Encoding: Let w{b) £ {1, . . . , 2"^} be the new index to be 
sent in block b, and assume that — 1) £ The encoder 
then selects u"-{w{b)\t{b)). At time i in block 6, upon receiving 
Si{b), the sender sends Xi{b) = f{ui{w{b)\t{b)), Si{b)). The 
relay will have an estimate w{b — 1) of the previous index 
w{b— 1). Assume that w{b — l) G T-,,-.- Then upon receiving 

s{b) the relay encoder sends xu = fi{uii{t{b)), Si{b)). 

Decoding: We assume that at the end of block b — 1, the 
receiver knows (u>(l), . . . , w{b — 2)) and . . . ,t{b ~ 1)) 

and the relay knows {w{l), . . . ,w{b — 1)) and consequently 
(t(l), . . . , t{b)). The decoding procedures at the end of block 
b are as follows: 

1) With {t{b) , yi (b) , s"(6)), the relay estimates the message 
of the transmitter as w{b) if there exists a unique w{b) such 
that {u"{iv{b)\t{b)),u'l{t{b)),y'^{b),s"{b)) are jointly typical. 



It can be shown that ■w{b) = w{b) with arbitrarily small 
probability of error, if 

R < I{U;Yi\Ui,S) 

and n sufficiently large. 

2) The receiver declares that t{b) was sent if there exists 
exactly one t{b) such that y" {b)) is jointly typical. It 
can be shown that t{b) = t{b) with arbitrarily small probability 
of error, if 

Ro<nUi;Y) (3) 

and n sufficiently large. 

3) Assuming that t{b) is decoded succesfully at the receiver, 
then w{b—l) is declared to be the index sent in block 6 — 1 if 
there is a unique w{b — 1) G Tt{b) that is jointly typical with 

— 1). It can be shown that if n is sufficiently large and 

if 

R < I{U;Y\Ui) + Ro (4) 

then w{b — 1) ^ w{b — l)x with arbitrarily small probability 
of error Combining (|3} and yields the condition R < 
IiU,U,-Y). 

Let us now prove the converse for any rate-i? code with 
encoding functions Xi{W,S') and Xu{S\Y'-^). We define 
the auxiliary random variables Ui = {W,Y''~^ , S'^~^) and 
Uu = {Y^^\S'''^). It is easy to see that {U^,Uu) is 
independent of Si and that Xi and Xu are deterministic 
functions of {Ui,Si) and (Uu^Si), respectively. We have 

/(I^;r") ^^I{W;Y,\Y'-^) 
1=1 

= Y,H{nY'-^) - H{Y,\Y'-\W) 

n 

< H{Y,) - H{Y,\Y'-\Yr\W, S'-^) 

n 

= Y,l{U,,Uu;Yi) 

i=l 

/(Vr;F") < /(W^;y",Fj^^|S'") 

n 

= Y,H{Y,,Yu\Y'-\Yl-\S^) 
-H{Y,,Yu\Y'-\Yl-\S^,W) 

n 

= Y,H{Y,,Yu\Y''-\Yr\S'^) 

1=1 

- H{Y,,Yu\Y'-\Yl-\S^, W, X„ X^) 

n 

<Y,H{Y,,Yu\Yl-\S'~\S,) 

1=1 

^ H{Yi, Yii\Xi, Xu, Si) 

n 

<J2h{y^,Yu\yi-\s'-\s,) 



~HiY,,Yi,\X,,Xu,S,,Y'~\Yl-\S'-\W) 

n 

< J2 H{Y,,Yu\Uu, S^) - H{Y,,Yu\U,, Uu, S,) 



i=l 



Y,I{U,-Y,,Yu\Uu,S,) 



Fano's inequality and the use of the usual time-sharing 
random variable will show 

R < mill {/([/, Ui;Y), /([/; Y,Yi\Ui,S)} + e„. 

Combining the following Markov relationship 

U ^{S,Ui,Yi)^Y 

with the degradedness of the channel and the fact that X and 
Xi are functions of {U, S) and {Ui,S), we can easily show 
that I{U;Y,Yi\Ui,S) ^ I{U;Yi\Ui, S). This completes the 
proof of the converse. ■ 

IV. Multiple Access Channels 

Definition 3: The discrete memoryless multiple access 
channel with state information consists of input alphabet 
Xi X X2, state alphabet S, output alphabet y, and a probability 
transition function p{y\xi, X2, s); see Figure |3 

Again we consider the case when the state variable S is 
causally available to the transmitters. 



Wi Xu(WuS-) 



H'2 " X2,{W2,S') 
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Fig. 4. Multiple access channel with state causally available to the 
transmitters. 



Define the region i?™/"^ > to be the convex hull of all rate 
pairs (i?i,i?2) satisfying 

Ri < HUi;Y\U2) 
R2<I{U2;Y\Ui) 
Ri + R2<I{Ui,U2;Y) 

over all p{ui,U2), xi — /i(ui,s), X2 ~ /(u2,s). Similarly, 
define i?™/"^ s , > to be the convex hull of all rate pairs 
, R2 ) satisfying the same set of inequalities over all 
p{ui,U2), xi = s), and X2 = f{u2, s). 

For the multiple access channel case, the capacity theorem 
is yet to be established, mostly because of the coupling of two 
auxiliary random variables in the proof of the converse. Here 
we give bounds on the capacity region instead. 

Theorem 3: Let C'™'^ denote the capacity region of the 
multiple access channel with state causally available at both 
transmitters. Then i?™'^ ^ , ^ C C""" C i?"^ ^ 

p{ui)p{U2) P(Ui,U2) 

Proof: Following the same arguments as before, it is 
not hard to show that C'™'"^ C i?"/"^ s; hence, we skip the 
details. 



For the lower bound, we first fix p{ui)p{u2) , s), and 

/2(W2,S). 

Codebook Generation. Generate 2"^^ independent code- 
words Ui{w\), wi G {1, 2, . . . , 2"^^}, generating each ele- 
ment i.i.d. ^ niLiPl"!*) ™d 2"^^ independent codewords 
^2(^2), W2 £ {1, 2, . . . , 2"^^}, generating each element 
i.i.d. ~ n"=i ?'('^2i)- These codewords form the codebook, 
which is revealed to the senders and the receiver 

Encoding. To send message indices wi G {1, 2, . . . , 2"^i} 
and W2 G {1, 2, . . . , 2"^^}, select the corresponding code- 
words Ui{wi) and U2{w2). At time i, upon receiving Si, 
transmitter 1 sends xi,i = fi{uii{wi) , Si) and transmitter 2 
acts similarly. 

Decoding. Let A^"^ denote the set of typical (J/f, U^, F") 
sequences. If there exists a unique pair Wi and W2 such that 
{U^{Wi),U^{W2),Y'') e A^"^ declare that {Wum) was 
sent, otherwise declare an error 

We now analyze the probability of error Define the events, 

E,,,={{U^{^),U^{J),Y-)eAi-^}. 

Without loss of generality assume the messages sent were 
Wi = 1 and W2 = 2, the probability of error becomes. 

From the AEP, P{Ef j^) ^ 0. For the second term we get 

P{E,^,) ^ P{(c/f W,t/2"(i),y") e 

E P«M«2,y") 

< E 2""'^-^'^^i)~'^)2~"'^-^'^'^^''^)"''^ 

(«r,«J,y")6A<"' 
<- 2n(-fi"(C/l,C/2,y)+e)2-"(-f^"('^i)-«)2~"(-f^('^2,i')-e) 

^ 2-n{H{Ui)+H{U2,Y)-H{Ui,U2,Y)-3e) 

^ 2""(-'"(^i'^2,l')-3e) 

^ 2-n{I(Ui;Y\U2)-3e) 

where the last equality follows since Ui and U2 are indepen- 
dent. Similarity for j ^ 1, 

<2-"(^(^^'^l^i)-3^), 

and for ^ (1,1) 

P(i^zj) < 2-"(^(^i'^^-^)-4^). 
We can now write the probabiUty of error as, 

p(") < P(^Ei i) + 2"-^i2^"^"^^'^"''^''^^^~^'^^ 

_|_ 2n-R22-"(-f('^2;l'|C/i)-3e) 

_|_ 2n(-Rl+-R2)2-"(^('^i.'^2;>')-4e) 

which tends to zero if the conditions of the theorem are met. 



V. Concluding Remarks 

We characterized the capacity region of a few simple 
multiple user channels with state information, in single-letter 
formulas. The value of this result may lie in expanding the set 
of toy examples in network information theory and thus giving 
a glimpse on the structure of the theory. Questions still remain 
on other multiple user channels and on the cost of causality 
of state information. 
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